An Efficient Approach for Assessing Parameter Importance in Bayesian Optimization

We describe a method for quantifying the importance of a blackbox function’s input parameters and their interactions, based on function evaluations obtained by running a Bayesian optimization procedure. We focus on high-dimensional functions with mixed discrete/continuous as well as conditional inputs, and therefore employ random forest models. We derive the first exact and efficient approach for computing efficient marginal predictions over subsets of inputs in random forests, enabling an exact quantification of parameter importance in the functional ANOVA framework. We demonstrate these techniques by assessing the importance of parameters in several recent applications of Bayesian optimization.